VRChoir: Exploring Remote Choir Rehearsals via Virtual Reality

Choral singing is a creative process that involves continuous, organized, nonverbal communication between conductors and singers. Since the COVID pandemic, choirs are moved to videoconferencing systems for rehearsals. However, the limitation of 2D video interfaces restricts the nonverbal communication, spatial awareness, and sense of presence in choral rehearsal. We designed, implemented, and evaluated VRChoir, a VR-based platform for choir rehearsals to improve these pain points. We evaluated VRChoir with conductors and singers with experience rehearsing in a remote environment. Our findings reveal that VR can be a starting point for improving the sense of presence and quality of non-verbal communication in remote music rehearsals. © 2023 IEEE.

Research to Enhance the Sense of Presence Based on the Positional Relationship Between the Sound Source and the Distance

In this research, through the survey and experiments, we examined whether the reflection of distance to sounds leads to an improvement in the sense of reality, and whether it is possible to enhance the sense of presence by reproducing the feeling of distance in environment that each of audience can easily build. The basic idea is from the needs of the times: the increase in online events due to the spread of COVID-19, and the loss of sense of distance and lack of realism when watching online events. From the survey results on ‘sense of reality' at the first stage of the research, it became clear that ‘sense of reality' is composed of multi kinds of sensory elements. Therefore, it is thought that, if sounds reflect sense of distance, it would lead to the enhancement of the ‘three-dimensional feeling' and the ‘sense of cause and effect', which are components of the ‘sense of presence', and thus this results in the enhancement of the sensory reality of "being there” when viewing a video. Also, the sound is attenuated by different kinds of causes, and in the large-scale venues where events take place, it is the case that the ‘sound attenuated by the distance from the sound source' reaches most of the seats. To reproduce a sense of distance, two kinds of experiments were conducted to confirm whether people recognize it and to check whether it is reproducible. From the results, it has been shown that the feeling of distance from the object can be perceived almost accurately by the audio information and the ‘distance perception' can be altered to some extent by the volume change. Recognizing distance leads to an improvement in the three-dimensional sense, which is one of components of the sense of reality, and it is thought that matching the feeling of distance perceived by sound and the feeling of distance perceived by the image can improve the ‘sense of cause and effect'. Therefore, based on the results of the experiment, it is considered possible to enhance the sense of presence by changing the sound volume in line with what is on the video. Even in a re-viewing environment that can be generally adjusted like when participating in an event online. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Human-robot interaction environment to enhance the sense of presence in remote sports watching

The recent coronavirus disease 2019 (COVID-19) outbreak has helped increase the popularity of online video communication and meeting platforms as alternatives to face-to-face interactions. Such a trend has also triggered the emergence of remote cheering systems, hinting at the possibility that people could enjoy watching sports games in virtual environments even from their homes in the near future. However, reproducing a sense of presence similar to the atmosphere felt by fans in stadiums or event venues is a major challenge within virtual environments. Thus, our idea is to embed groups of cheerful robots in virtual environments, thereby creating a sense of unity and mimicking emotion spread among fans. In this study, we built a virtual cheering environment and embedded a game-event driven behavior model that enables a group of robots to display various emotions through nonverbal reactions according to the game flow. Then, we conducted a preliminary evaluation of the proposed system, where the participants and a group of robots were placed in a virtual cheering environment to watch a baseball game. The obtained results hinted at the meaningfulness of the proposed approach. Nevertheless, further work is necessary to achieve a sufficient sense of presence and validate the effectiveness of our proposed environment. © 2022 IEEE.

Facilitated Tabletop Games in a Mediated Environment

Tabletop games that require the aid of a human facilitator are typically designed for a physical environment. However, during the COVID-19 pandemic, lockdown measures prevented people from gathering. Therefore, tabletop games were sometimes played and facilitated in a mediated environment using online communication tools instead. But this setting possibly deteriorates the players’ game experience. To understand the effect of playing facilitated tabletop games in a mediated environment we measured the player experience of the game Cue Kitchen in a mixed-method study comparing physical and mediated game sessions. Forty-four players played eleven game sessions, three in a physical environment and eight in a mediated environment. Of all seven dimensions of game experience measured, only one differed significantly between the two experimental conditions: players in a mediated environment became significantly more tired than players in a physical environment. The qualitative results explained why: players in a physical setting can wander off, while players in an online setting have to stay focused on their screen and, therefore, grow more tired. The research results suggest that facilitated tabletop games may be played in a mediated environment instead of a physical environment, without significant loss of player experience. © 2022, Springer Nature Switzerland AG.

Stage Together: Remote Rehearsal of Theater Blocking

The COVID-19 pandemic has forced many theater productions to shift to remote rehearsals and performances using online video conference platforms. Video conference software makes it possible to share individual performances, but loses the sense of presence together and provides no way to visualize blocking movements on the shared space of the stage. Rehearsal by video conference can cause productions to either omit blocking altogether or have only a few on stage rehearsals, sometimes just days before opening. This paper presents a Web-based system that enables productions to remotely rehearse blocking at any stage of the production. Remote performers can propose their own blocking, rehearse scripted cues and visualize movements in a shared virtual stage. This paper describes the design, implementation, and focus group evaluations of Stage Together with a college theater production of Medea by Euripides. © 2022 ACM.

Excitement Projector: Augmenting Excitement-Perception and Arousal through Bio-signal-based Haptic Feedback in Remote-SportWatching

For non-sport fans, perceiving the excitement of surrounding fan groups and the arousal of collective emotions are some of the crucial factors that motivate their engaged excitement and loyalty in a sport;these factors are closely related to the process of evolution from a non-fan to a fan. The global COVID-19 pandemic changed the way of sport-watching from watching at the arena to watching from home alone. This has highlighted significant difficulties in the excitement transmission and arousal channel between non-sport fans and fans. Previous remote emotional intervening mediums had been limited to the use of virtual avatars to convey partners' external cues (such as appearance), to enhance the sense of presence from visual-audio perspectives. In this study, we explored a novel remote emotional intervening medium that conveys sport-fans' internal cues (bio-signals) that are widely believed to be related to internal emotional states of human, and displays those signals in a way that gives non-sport fans a deeper and more immersive experience: haptic feedback experience. Three bio-signal-based haptic feedback prototypes were developed, including heart-rate-based vibration, electromyography (EMG)-based pressure, and skin-temperature-based thermal feedbacks. An exploratory pilot study was conducted on a group of non-sport fans in a lab-control environment of remote-sport-watching to explore the effectiveness of the proposed mediums in enhancing their perception of sport-fan's excitement (emotion perception). Besides, we also analyzed non-sport fans' heart rate data when they were participants in the experiments to measure the performance of the proposed mediums in evoking the engaged excitement of non-sport fans (emotion arousal). Our results indicate the outstanding ability of EMG-based pressure feedback in effectively enhancing the process of emotion perception and the notable advantage of heart-rate-based vibration feedback in the arousal of non-sport fans' engaged excitement. This study presents the potential utility of bio-signal-based haptic feedback in augmenting remote emotional perception and arousal and also provides the underlining support for the future exploration and development directions for social computing based on bio-signals and haptic technologies. © 2022 ACM.

Mixed Reality Ulysses

'MR Ulysses' is a creative project investigating the possibilities of live performance using volumetric video (VV) techniques displayed via VR and AR technologies. It explores the opening scene of James Joyce's acclaimed modern masterpiece of twentieth-century literature, 'Ulysses'. Set in 1904, 'Ulysses' is noted for its geographic specificity, and every year Joyce scholars and enthusiasts ritualis-tically visit Dublin on June 16th to celebrate an annual literature pilgrimage called "Bloomsday."The tradition involves dressing up in the fashion of the day, retracing the steps of the central characters by visiting many of the sites that are mentioned, and eating, drinking, and re-enacting scenes from the book. Since the Covid-19 pandemic, the global culture industry of live performance events all but collapsed. The impact on theatre performers, artists, audiences, and stakeholders has been substantial. Bloomsday usually attracts thousands of literature tourists to Ireland's capital city to engage in performances at multiple locations around Dublin;however, in 2020, it was canceled due to the pandemic. In 2021 there was a better effort to hold online events, but they lacked the sense of place and presence so crucial to Bloomsday. This sociological turn has occasioned a profound new emphasis on the need for theatre/performance practitioners to explore the immersive potential of AR/VR technologies because the shortcomings of simple videoconferencing/webcasting are obvious for theatre audiences. 'MR Ulysses' is the first part of a series of creative-cultural experiments investigating questions around preservation, access, reactivation, and the transmission of dramatic/literary heritage in the twenty-first century. Captured using cuttingedge VV techniques, MR Ulysses enables audiences from any part of the globe to experience a live-action Bloomsday ritual in a site-specific 3D simulation via a VR headset. The goal is to suggest new horizons for the performing arts in the context of interactive digital media technologies. © 2021 Owner/Author.

Hovering and Contact Representation of Laser Contour-Based Hand with Swinging Tablet PC for Distant Communication

As covid-19 spread worldwide, the growing trend to reduce people's mobility and the establishment of new lifestyles such as remote work, contactless communication systems have been required. Using a projected hand makes it possible to realize remote communication with a sense of presence without loading a user's physical body. However, conventional projected hand systems have a problem that the relative brightness of the hand image in a bright living room is low. Therefore, we propose a configuration that combines a video call system and a contour-based projected hand by a laser scanning projector. In addition, we incorporate a haptic feedback system with a tablet PC to improve the sense of contact with remote objects. To enhance the sense of contact in remote areas, we focus on providing the transition between hovering and contact states of the contour-based projected hand and users' hands. According to the experiment, we confirmed that there is a possibility to express hovering and contact even with simple swinging flat-panel feedback and visual effects of the contour-based projected hand. © 2022 Owner/Author.

Improving online learning interactivity with 3D virtual classroom models

The world of education is experiencing a very significant impact of the Covid-19 pandemic. Online learning is a solution in overcoming learning limitations due to social distancing. Researchers developed a 3d virtual classroom using a Minecraft model to increase online learning interactivity. The 3 dimensional virtual classroom model design that has been developed has received respondents' acceptance of 90.00% in the "Very Eligible" category. Testing the 3 virtual classroom features feature got a percentage of 90.83% with the "Very Eligible" category to produce a sense of presence. In the interactivity test, character interaction is 89.17% in the "Very Eligible" category. The effectiveness of online learning continuity using the 3D virtual classroom model tested gets 89.17% with the "Very Eligible" category. In addition, the test uses an average score of SUS (System Usability Scale) of 73.67, so it can be concluded that the 3D virtual classroom model is "acceptable" as an online learning medium. © 2021 Institute of Physics Publishing. All rights reserved.

X3D Field Trips for Remote Learning

Combinations of immersive media and graphic portrayals can enable the human subjective sense of Presence. This paper collects our experiences and evaluations from six projects that use Extensible 3D (X3D) interactive graphics to deliver spatial experiences across the WWW. X3D enables the combination of spherical panoramas with 3D models and maps to visually transport users to a specific real location at a specific time. Remote users have access to these worlds through a Web-browser or other immersive device;local users in a CAVE can collaborate with natural physical gestures;. We reflect on the graphical and interactive requirements of these projects and provide guidance for future applications. In the face of physical lock-downs and distancing due to the CoVID pandemic, such platforms illustrate the opportunities and challenges in the design and delivery of spatial visualizations, especially for remote learning. © 2021 ACM.